<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Iterators and Generators</title>
<link rel="stylesheet" href="/cfg/format.css" type="text/css">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="keywords" content="Python, iterators, generators, programming, language, learn Python">
<meta name="description" content="In this part of the Python tutorial, we prese t
iterators and generators.">
<meta name="language" content="en">
<meta name="author" content="Jan Bodnar">
<meta name="distribution" content="global">

<script type="text/javascript" src="/lib/jquery.js"></script>
<script type="text/javascript" src="/lib/common.js"></script>

</head>

<body>

<div class="container">

<div id="wide_ad" class="ltow">
<script type="text/javascript"><!--
google_ad_client = "pub-9706709751191532";
/* 160x600, August 2011 */
google_ad_slot = "2484182563";
google_ad_width = 160;
google_ad_height = 600;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
</div>

<div class="content">

<a href="/" title="Home">Home</a>&nbsp;
<a href="..">Contents</a>


<h1>Iterators and Generators</h1>

<p>
According to Wikipedia, an <b>iterator</b> is an object 
which allows a programmer to traverse through all the elements of a collection,
regardless of its specific implementation.
</p>

<script type="text/javascript"><!--
google_ad_client = "pub-9706709751191532";
/* NewSquare */
google_ad_slot = "0364418177";
google_ad_width = 300;
google_ad_height = 250;
//-->
</script> 
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> 
</script> 

<p>
In Python programming language, an iterator is an object which implements the 
iterator protocol. The <b>iterator</b> protocol consists of 
two methods. The <code>__iter__()</code> method, which must return 
the iterator object and the <code>next()</code> method, which returns 
the next element from a sequence. 
</p>


<p>
Python has several built-in objects, which implement the iterator protocol. 
For example lists, tuples, strings, dictionaries
or files. 
</p>

<pre class="code">
#!/usr/bin/python

# iter.py


str = "formidable"

for i in str:
   print i,

print

it = iter(str)

print it.next()
print it.next()
print it.next()

print list(it)
</pre>

<p>
In the code example, we show a built-in iterator on a string. In Python a 
string is an immutable sequence of characters.
The <code>iter()</code> function returns an iterator on object.
 We can also use the <code>list()</code>
or <code>tuple()</code> functions on iterators.  
</p>

<pre>
$ ./iter.py 
f o r m i d a b l e
f
o
r
['m', 'i', 'd', 'a', 'b', 'l', 'e']
</pre>


<p>
Iterators have several advantages:
</p>

<ul>
    <li>Cleaner code</li>
    <li>Iterators can work with infinite sequences</li>
    <li>Iterators save resources</li>
</ul>

<p>
By saving system resources we mean, that when working with iterators, we 
can get the next element in a sequence without keeping the entire dataset in memory.
</p>

<pre class="code">
#!/usr/bin/python

# wantme1.py

f = open('ifyouwantme', 'r')

while True:
   line = f.readline()
   if not line: break
   else: print line,

f.close()
</pre>

<p>
This code prints the contents of the ifyouwantme file. 
</p>

<pre class="code">
#!/usr/bin/python

# wantme2.py

f = open('ifyouwantme', 'r')

for line in f:
   print line,

f.close()
</pre>

<p>
The wantme2.py script does the same. In this case, we use iterators. The code is cleaner. 
</p>

<p>
In the following example, we create our own object that will implement the iterator protocol.
</p>

<pre class="code">
#!/usr/bin/python

# iterator.py


class seq:
   def __init__(self):
      self.x = 0

   def next(self):
      self.x += 1
      return self.x**self.x

   def __iter__(self):
      return self


s = seq()
n = 0

for i in s:
   print i
   n += 1
   if n > 10:
      break
</pre>

<p>
In the code example, we create a sequence of numbers 1, 4, 27, 256, ... . 
This demonstrates that with iterators, we can work with infinite sequences. 
The for statement calls the <code>iter()</code> function on the 
container object. The function returns an iterator object that defines the 
method <code>next()</code> which accesses elements in the container one at a time.
</p>

<pre class="explanation">
 def next(self):
    self.x += 1
    return self.x**self.x
</pre>

<p>
The <code>next()</code>  method returns the next element of a sequence. 
</p>

<pre class="explanation">
 def __iter__(self):
    return self
</pre>

<p>
The <code>__iter__</code> method returns the iterator object. 
</p>

<pre class="explanation">
 if n > 10:
      break
</pre>

<p>
Because we are working with an infinite sequence, we must interrupt the for loop. 
</p>

<pre>
$ ./iterator.py 
1
4
27
256
3125
46656
823543
16777216
387420489
10000000000
285311670611
</pre>

<p>
The loop can be interupted in another way. In the class definition we must 
raise a <code>StopIteration</code> exception. In the following 
example, we redo our previous example. 
</p>

<pre class="code">
#!/usr/bin/python

# stopiter.py


class seq14:
   def __init__(self):
      self.x = 0

   def next(self):
      self.x += 1
      if self.x > 14:
         raise StopIteration
      return self.x**self.x

   def __iter__(self):
      return self


s = seq14()

for i in s:
   print i
</pre>

<p>
The code example will print first 14 numbers of a sequence. 
</p>

<pre class="explanation">
if self.x > 14:
    raise StopIteration
</pre>

<p>
The <code>StopIteration</code> exception will cease the for loop. 
</p>



<h2>Generators</h2>

<p>
In general, a <b>generator</b> is a special routine that can 
be used to control the iteration behaviour of a loop. A generator is similar 
to a function returning an array. A generator has parameters, it can be called 
and it generates a sequence of numbers. But unlike functions, which <b>return</b> 
a whole array, a generator <b>yields</b> one value at a time.
This requires less memory. (Wikipedia)
</p>

<p>
Generators in Python:
</p>

<ul>
    <li>Are defined with the def keyword</li>
    <li>Use the yield keyword</li>
    <li>May use several yield keywords</li>
    <li>Return an iterator</li>
</ul>

<p>
Let's look at an generator example. 
</p>

<pre class="code">
#!/usr/bin/python

# generator.py


def gen():
   x, y = 1, 2
   yield x, y
   x += 1
   yield x, y


it = gen()

print it.next()
print it.next()

try:
   print it.next()
except StopIteration:
   print "Iteration finished"
</pre>

<p>
As we can see, a generator is defined with a <code>def</code> 
keyword, just like normal functions. 
We use two <code>yield</code> keywords inside the body of a 
generator. Now it is important to understand, how actually the yield keyword works. 
It exits the generator and returns a value. Next time the <code>next()</code>
function of an iterator is called, we continue on the line following the yield 
keyword. Note that the local variables are preserved throughout the iterations. 
When there is nothig left to yield, a <code>StopIteration</code> 
exception is raised. 
</p>

<pre>
$ ./generator.py 
(1, 2)
(2, 2)
Iteration finished
</pre>

<p>
The following example we will calculate fibonacci numbers. The first number of 
the sequence is 0, the second number is 1, and each subsequent number is equal 
to the sum of the previous two numbers of the sequence itself.
</p>

<pre class="code">
#!/usr/bin/python

# fibonacci.py

import time
import sys

def fib():
   a, b = 0, 1
   while True:
      yield b
      a, b = b, a + b


iter = fib()

try:
   for i in iter:
      print i,
      time.sleep(1)
      sys.stdout.flush()
except KeyboardInterrupt:
   print "Calculation stopped"
</pre>

<p>
The script will continuously print fibonacci numbers to the console. 
</p>


<p>
In this chapter, we have covered iterators and generators in Python.
</p>


<div class="botNav, center">
<span class="botNavItem"><a href="/">Home</a></span> ‡ <span class="botNavItem"><a href="..">Contents</a></span> ‡ 
<span class="botNavItem"><a href="#">Top of Page</a></span>
</div>


<div class="footer">
<div class="signature">
<a href="/">ZetCode</a> last modified May 20, 2008  <span class="copyright">&copy; 2007 - 2013 Jan Bodnar</span>
</div>
</div>

</div> <!-- content -->

</div> <!-- container -->

</body>
</html>

